Path ensembles averages in systems driven far-from-equilibrium 
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The Kawasaki nonlinear response relation, the transient fluctuation theorem, and the Jarzynski 
nonequilibrium work relation are all expressions that describe the behavior of a system that has 
been driven from equilibrium by an external perturbation. In contrast to linear response theory, 
these expressions are exact no matter the strength of the perturbation, or how far the system has 
been driven away from equilibrium. In this paper I show that these three relations (and several other 
closely related results) can all be considered special cases of a single theorem. This expression is 
explicitly derived for discrete time and space Markovian dynamics, with the additional assumptions 
that the single time step dynamics preserve the appropriate equilibrium ensemble, and that the 
energy of the system remains finite. 
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I. INTRODUCTION 

If a system is gently driven from equilibrium by a small 
time-dependent perturbation, then the response of the 
system to the perturbation can be described by linear re- 
sponse theory. On the other hand, if the system is driven 
far-from-equilibrium by a large perturbation then linear 
response, and other near-equilibrium approximations, are 
generally not applicable. However, there are a few re- 
lations that describe the statistical dynamics of driven 
systems which are valid even if the system is driven far- 
from-equilibrium. These include the Jarzynski nonequi- 
librium work relation which gives equilibrium free 
energy differences in terms of nonequilibrium measure- 
ments of the work required to switch from one ensemble 
to another; the Kawasaki relation 1^-^, which specifies 
the nonlinear response of a classical system to an arbi- 
trarily large perturbation; and a group of relations that 
can be collectively called "entropy production fluctuation 
theorems" p0|-p6||. I will specifically consider the tran- 
sient fluctuation theorem of Evans and Searles [ p]Jl4| 
which deals with entropy production of driven systems 
that are initially in equilibrium. The Gallavotti-Cohen 
p^ , p^ fluctuation theorem addresses entropy production 
in nonequilibrium steady-states, and will not be consid- 
ered in this paper. 

The relations listed above have been derived for a wide 
range of deterministic and stochastic dynamics. How- 
ever, the various expressions and applicable dynamics 
have several commonalities: the system starts in ther- 
mal equilibrium, it is driven from that equilibrium by 
an external perturbation, the energy of the system is fi- 
nite, the dynamics are Markovian, and if the system is 
unperturbed then the dynamics preserve the equilibrium 
ensemble. In this paper, it will be shown that these con- 
ditions are sufficient to derive the far-from-equilibrium 
expressions mentioned above. Indeed they can all be con- 
sidered special cases of a single theorem: 



(1) 



Here, indicates the average of the path function 

Path functions (such as the heat and work) are func- 
tionals of the trajectory that the system takes through 
phase-space. An average of a path function is implic- 
itly an average over a suitably defined ensemble of paths. 
In this paper, the path ensemble is defined by the ini- 
tial thermal equilibrium and the process by which the 
system is subsequently perturbed from that equilibrium. 
The left side of the above relation is simply averaged 
over the ensemble of paths generated by this process. We 
arbitrarily label this the forward process (subscript 'F'). 

For every such process that perturbs the system from 
equilibrium we can imagine a corresponding reverse per- 
turbation (subscript 'R'). We shall construct this process 
by insisting that it too starts from equilibrium, and by 
considering a formal time reversal of the dynamics. The 
right side of Eq. (|^) is JF, the time reversal of averaged 
over the reverse process, and weighted by the exponen- 
tial of /SWd- Here /3 = 1/kiT, T is the temperature of 
the heat bath, kg is Boltzmann's constant, and Wd is the 
dissipative work. The dissipative work is a path func- 
tion and is defined as Wd = W — Wr, where W is the 
total work done on the system by the external perturba- 
tion and Wr is the reversible work, the minimum average 
amount of work required to perturb the system from its 
initial to its final ensemble. 

In summary, Eq. (|^) states that an average taken over 
an ensemble of paths, which is generated by perturbing 
a system that is initially in equilibrium, can be equated 
with the average of another, closely related quantity, av- 
eraged over a path ensemble generated by the reverse 
process. This relation is valid for systems driven arbi- 
trarily far-from-equilibrium, and several other far-from- 
equilibrium relations can be derived from it. It is suf- 
ficient that the dynamics are Markovian, preserve the 
equilibrium ensemble, and that the energy of the system 
is finite. In the next section I derive from these conditions 
that such a system is microscopically reversible, Eq. (^) , 
in a sense that will be made precise. (This derivation is 
somewhat more general than that given previously [Q.) 
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The path ensemble average, Eq. (||), is an almost trivial 
identity given that the dynamics satisfy this condition. 
This derivation is given is Sec. (3), and various special 
cases are considered. 



II. MICROSCOPIC REVERSIBILITY OF 
DRIVEN SYSTEMS 

Let us consider a classical system which can exchange 
energy with a constant temperature heat bath, and which 
has a finite set of states, cc € {1, 2, 3, • • ■ , N}. The ener- 
gies of the states of the system are given by the vector 
E. If these state energies do not vary with time then 
the stationary probability distribution, tt, is given by the 
canonical ensemble of equilibrium statistical mechanics; 



p(a;|/3,E) =7r. 



exp{pF-f3E,}. 



(2) 



In this expression the sum is over all states of the system 
and E) = -P'^ In J2x e^p{-PE4 is the Helmholtz 
free energy of the system. 

In contrast to an equilibrium ensemble, the probability 
distribution of a nonequilibrium ensemble is not deter- 
mined solely by the external constraints, but explicitly 
depends on the dynamics and history of the system. Let 
us consider a stochastic dynamics with a discrete time 
scale, t G {0, 1, 2, 3, • ■ • , r}. The state of the system at 
time t is x(t), and the path, or trajectory that the sys- 
tem takes through this state space can be represented by 
the vector x = (a;(0), x(l), x(2), • • • , x{t)) . We make the 
assumption that the dynamics are Markovian j2^. This 
implies that the probability of making a transition be- 
tween states in a particular time step depends only on 
the current state of the system, and not on the previous 
history. The single time step dynamics are determined 
by the transition matrix M(t) whose elements are the 
transition probabilities; 



= P{x(t) ^ x{Ul)) 



(3) 



A transition matrix, AI, has the properties that all ele- 
ments must be nonnegative and that all columns sum to 
1 due to the normalization of probabilities: 



Mij > 



for all i and j, 
for all j. 



Let p{t) be a (column) vector whose elements are the 
probability of being in state i at time t. Then the single 
time step dynamics can be written as 



(4) 



(5) 



The state energies E(t) and the transition matrices 
M{t) are functions of time due to the external pertur- 
bation of the system, and the resulting Markov chain is 
non-homogeneous in time [^ . The vector of transition 
matrices M = (M (0), Af(2), • • • , M(t-1)) completely de- 
termine the dynamics of the system. We place the fol- 
lowing additional constraints on the dynamics; that the 
state energies are always finite (this avoids the possibility 
of an infinite amount of energy being transferred from or 
to the system), and that the single time step transition 
matrices must preserve the corresponding canonical dis- 
tribution. This canonical distribution, Eq. (||), is deter- 
mined by the temperature of the heat bath and the state 
energies at that time step. We say that the transition 
matrix is balanced, or that the equilibrium distribution 
iT{t) is an invariant distribution of M{t). 

TT{t) ^ M{t) Tr{t) 

Essentially this condition says that if the system is al- 
ready in equilibrium (given E(t) and /3), and the system 
is unperturbed, then it must remain in equilibrium. 

It is often convenient to impose the much more restric- 
tive condition of detailed balance. 



(6) 



In particular many Monte-Carlo simulations are detailed 
balanced. However, it is not a necessity in such simu- 
lations IH], and it is not necessary here. It is sufficient 
that the transition matrices are balanced. 

Each time step of this dynamics can be separated into 
two distinct substeps. At time t — the system is in 
state a;(0) with energy E{0)x(o)- In the first substep the 
system makes a stochastic transition to a state x(l) which 
has energy E(0)x(i). This causes an amount of energy, 
^'(0)a;(i) —£'(0)2^(0), to enter the system in the form of heat. 
In the second substep the state energies change from E(0) 
to E(l) due to the external perturbation acting on the 
system. This requires an amount of work, E{l)x(i) — 
E{0)x(i) ■ This sequence of substeps repeats for a total of t 
time steps. The total heat exchanged with the reservoir, 
Q, the total work performed on the system, W, and the 
total change in energy, AE, are therefore 



r— i 

Q[x] = - 



t=o 

r-l 



or equivalently as 



(7) 

(8) 

: E{T)x^^, - E{OU„, =W+Q. (9) 

The reversible work, W, ^ AF = F(/3,E(t)) - 
F(/3, E(0)), is the free energy difference between two 



AE 
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equilibrium ensembles. It is the minimum average 
amount of work required to change one ensemble into 
another. The dissipative work, VVd[x] = yV[x] — Wr, is 
defined as the difference between the actual work and 
the reversible work. Note that the total work, the dissi- 
pative work and the heat are all path functions. In this 
paper they are written with script letters, square brack- 
ets and/or as functions of the path, x, to emphasize this 
fact. In contrast AE is a state function; it depends only 
on the initial and final state. 

Now we will consider the effects of a time reversal on 
this Markov chain. In many contexts a time reversal 
is implemented by permuting the states of the system. 
For example, in a Hamiltonian system a time reversal 
involves inverting the momenta of all the particles. How- 
ever, it is equivalent, and in the current context much 
more convenient, to apply the effects of the time reversal 
to the dynamics rather than the state space. Thus the 
time-reversed trajectory, x, is a simple reordering of the 
forward trajectory; x{t) = x{T-t) and E(i) — E(r-t). 

We can derive the effect of a time reversal on a transi- 
tion matrix by considering a time homogeneous Markov 
chain. Let tt be the invariant distribution of the time- 
independent transition matrix M, given by the equilib- 
rium canonical ensemble. If the system is in an equilib- 
rium ensemble then a time reversal should have no effect 
on that ensemble, and the probability of observing the 
transition i — > j in the forward chain should be the same 
as the probability of observing the transition j — > z in the 
time-reversed chain. Because the equilibrium probability 
distribution is the same for both chains it follows that 

Mji T^i = Afjj T^j for all i,j . (10) 

In matrix notation this may conveniently be written as 

M = diag(7r)"^M^diag(7r). 

Here, diag(7r) indicates a matrix whose diagonal elements 
are given by the vector tt. M is referred to as the reversal 
of M nil, or as the 7r-dual of Af If the transition 

matrix obeys detailed balance, Eq. (||), then M — M. 

It is easy to confirm that M is a transition matrix; 
all entries are nonnegative because all equilibrium and 
transition probabilities are nonnegative, and all rows sum 
to 1, 



for all 



Further, we can demonstrate that M and M have the 
same invariant distribution. 



For the non-homogeneous chain the time reversal of 
the vector of transition matrices, M, is defined as 



M(t) = di&g{TT{T~t))-^M{T-tfdi&g{TT{T-t)). (11) 

The time reversal operation is applied to each transition 
matrix, and their time order is reversed. Note that for 
the transition matrices of the reverse chain the time in- 
dex runs from 1 to r, rather than to t-1. Therefore, 
M{t) is the transition matrix from time t to time t+1 (see 
Eq. (P), but M(t) is the transition matrix from time t-1 
to time t. 



m^M{t)p{t-i). 



(12) 



This convention is chosen so that the time indexes of the 
various entities remains consistent. Thus for the reverse 
chain at time t the state is x{t), the states energies are 
E(t) and the corresponding equilibrium distribution is 
7f(f), which is an invariant distribution of M{t). 

Another consequence of the time reversal is that the 
work and heat substeps are interchanged in the reverse 
chain. The heat, total work and dissipative work are 
all odd under a time reversal: Q[x] — — Q[x], yV[x] — 
-W[x] and yVd[x] = -yVd[x]. The total change in en- 
ergy, and the free energy change are also odd under a 
time reversal, but to avoid ambiguity a 'A' always refers 
to a change measured along the forward process. 

We are now in a position to prove an important sym- 
metry for the driven system under consideration. Let 
T'lx I a;(0), M] be the probability of the trajectory x, 
given that the system started in state a;(0). The probabil- 
ity of the corresponding reversed path is V[^ \ iE(0), M]. 
The ratio of these path probabilities is a simple function 
of the heat exchanged with the bath. 



7^[x|x(0),M] 
P[x|J(0),M] 



exp{-/3Q[x]}. 



(13) 



At the risk of ambiguity, a system with this property will 
be described as microscopically reversible [| 22 2^. 

We proceed by expanding the path probability as a 
product of single time step transition probabilities. This 
follows from the condition that the dynamics are Marko- 
vian. 



P[x|x(0),M] 



\{p{x{t) x{Ul)) 
t=o 

Y[P{x{t')^x{t'.l)) 



t'=0 



For every transition in the forward chain there is a 
transition in the reverse chain related by the time re- 
versal symmetry, Eq. (|l|). The path probability ratio 
can therefore be converted into a product of equilibrium 
probabilities. 



7'[x|S(0),M] 
P[X |x(0),M] 



n 



7r(i). 



T-1 

n 

t=0 



p{x{t+l)\f3,E{t)) 
p{x{t)\p,Eit)) 
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T— i 



t=o 



= exp{-/?Q[x]} 

The second line follows from the definition of the canon- 
ical ensemble, Eq. (|^), and the final line from the defini- 
tion of the heat, Eq. (0). 

The essential assumptions leading to this condition 
of microscopic reversibility are that the state energies 
are always finite, and that the dynamics are Markovian, 
and if unperturbed preserve the equilibrium distribution. 
These conditions are valid independently of the strength 
of the perturbation, or the distance of the ensemble from 
equilibrium. The extension to continuous time and con- 
tinuous phase-space appears straightforward, although it 
is technically more difficult to be completely rigorous. 
However, Jarzynski ||2^ has recently demonstrated that 
deterministic Hamiltonian system coupled to many heat 
baths are also microscopically reversible. 



III. PATH ENSEMBLE AVERAGES 

We are now in a position to consider the path ensem- 
ble average (Eq. {^) detailed in the introduction. A 
system that is initially in thermal equilibrium is driven 
away from that equilibrium by an external perturbation, 
and the path function T[x\ is averaged over the resulting 
nonequilibrium ensemble of paths. The probability of a 
trajectory is determined by the equilibrium probability 
of the initial state, and by the vector of transition matri- 
ces that determine the dynamics. Therefore, the average 
of over the ensemble of trajectories can be explicitly 
written as 

(•^)f = E E(0)) P[x I x{0),M] ^[x] . 

X 

The sum is over the set of all paths connecting all pos- 
sible initial and final states. Given that the system is 
microscopically reversible it is a simple matter to con- 
vert the above expression to an average over the reverse 
process. We first note that 

p(:r(0)|/3,E(0))P[x|a:(0),M] ^ ^+^Ai<;-/3AF-/5Q[x] 
p{x{Q)\P,E{0))n^\W),M] 



^ g+/3W[x]-/3AF 



,+/3Wd[x] 



(14) 



The first line follows from the condition of microscopic 
reversibility Eq. (|l^) , and the definition of the canonical 
ensemble, Eq. (j^ . Recall that AF is the reversible work 
of the forward process, and that Wd[x] is the dissipative 
work. The set of reverse trajectories is the same as the 
set of forward trajectories, and we define .F[x] = ^[^]- 



Therefore, 

(•^)f = E/'(^(0)|/^' E(0)) P[i\ M]Tm e-'^^-t^] 

It is frequently convenient to rewrite Eq. (|l|) as 

{:Fe-'^^^)^ = {T)^, (15) 
where !F has been replaced with Te^^^'^, and T with 

A. Jarzynski nonequilibrium work relations 

A variety of previous known relations can be consid- 
ered special cases or approximations of this nonequilib- 
rium path ensemble average. In the simplest case we 
start with Eq. (p^, and then set T — J- — 1 (or any 
other constant of the dynamics). Then 



1. 



(16) 



The right side is unity due to normalization of probabil- 
ity distributions. We are now taking an average over a 
single path ensemble, and the remaining subscript, "F", 
becomes superfluous. The dissipative work, Wd can re- 
placed by W — AF, and the change in free energy can be 
moved outside the average since it is path independent. 
The result is the Jarzynski nonequilibrium work relation 

mm. 



-f3W\ 



(17) 



This relation states that if we convert one system into 
another by changing the energies of all the states from an 
initial set of values to a final set of values over some finite 
length of time, then the change in the free energies of the 
corresponding equilibrium ensembles can be calculated 
by repeating the switching process many times, each time 
starting from an equilibrium ensemble, and taking the 
above average of the amount of work required to effect the 
change. In the limit of instantaneous switching between 
ensembles, (we change the energies of all the states in a 
single instantaneous jump) this relation is equivalent to 
the standard thermodynamic perturbation method that 
is frequently used to calculate free energy differences by 
computer simulation 

It is possible to extend Eq. ( p^ ) to a more general class 
of relations between the work and the free energy change 
||. Suppose that T = /(W) where /(W) is any fi- 
nite function of the work. Then T = /(-W), because 
the work is odd under a time reversal. Inserting these 
definitions into Eq. (Q) and rearranging gives 



-l3AF 



(/(+>v))p 



(/(-W)e-/'W)^ 



(18) 
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Recall that AF is defined in terms of the forward pro- 
cess. Suppose that we have obtained rip independent 
measurements of the work required for the forward pro- 
cess, and rin independent measurements from the reverse 
process. An interesting question is what choice of /(W) 
leads to the highest statistical accuracy for AF. For 
instantaneous switching this question was answered by 
Bennett in his derivation of the acceptance ratio 

method for calculating free energy differences. For finite 
time switching Bennett's derivation can be followed al- 
most line for line. We therefore omit the details, and 
simply record the conclusions in the present notation. 
The least statistical error will result if we take J- — 
(l+exp{+/3>V+C})-i, and T = (l+exp{-/3W+C})-i. 
Then 



-/3AF 



((l+exp{+/3>V+C})-i)p 



exp{+C} . (19) 



The optimal choice of the constant C is -pAF+lnrip/n^. 
This relation must be solved self-consistently, since AF 
appears on both sides. 



B. Transient fluctuation theorem 

Another interesting application of the path ensemble 
average is to replace the finite function of the work used 
above with a S function, !F = 5(/3Wd-/3yVd[x[), — 
(5(/3>Vd+/3>Vd[x]). Plugging these rs into Eq. (g) gives 

(J(/3Wd-/3Wd[x]) e-''^<^)p = (<5(/3>Vd+/3Wd[x]))j, , 

or i?(+/3>Vd) e^^^- = P^{-PWa) ■ 

Here, U{+(3Wd) is the probability of expending the 
specified amount of work in the forward process, and 
ii^(-/3Wd) is the probability of expending the negative 
of that amount of work in the reverse process. If 
i^(-/3>Vd) 7^ then we can rearrange this expression as 



S(-/3Wd) 



(20) 



The system of interest starts in equilibrium and is per- 
turbed for a finite amount of time. If it is allowed to re- 
lax back to equilibrium then the change in entropy of the 
heat bath will be -/3Q, and the change in entropy of the 
system will be f3AE~pAF. Therefore, the total change 
in entropy of the universe resulting from the perturbation 
of the system is -(3Q+13AE-I3AF = f3W-(3AF = /3Wd, 
the dissipative work. Thus Eq. ( |20| ) can be interpreted 
as an entropy production fluctuation theorem. It relates 
the distribution of entropy productions of a driven sys- 
tem that is initially in equilibrium to the entropy pro- 
duction of the same system driven in reverse. As such 
it is closely related to the transient fluctuation theorems 
of Evans and Searles llTl|,|l4|. The connection between 



this fluctuation theorem, the Jarzynski nonequilibrium 
work relation and microscopic reversibility was originally 
presented in 



C. Kawasaki response and nonequilibrium 
distributions 

All of the above relations were derived from Eq. (|l|) 
by inserting a function of the work. Another group of 
relations can be derived by instead setting .7-" to be a 
function of the state of the system at some time. In 
particular if we average a function of the final state in the 
forward process, J- = f{x{T)) , then we average a function 

of the initial state in the reverse process, J- = f{x{Q)): 

Therefore, in the reverse process the average is over the 
initial equilibrium ensemble of the system, and the sub- 
sequent dynamics are irrelevant. We can once more drop 
reference to forward or reverse processes, and instead use 
labels to indicate equilibrium and nonequilibrium aver- 
ages: 



-/3Wd 



)„ea = (/(^W))e, 



(21) 



This relation (also due to Jarzynski j32|) states that the 
average of a state function in a nonequilibrium ensemble, 
weighted by the dissipative work, can be equated with 
an equilibrium average of the same quantity. 

Another interesting relation results if we insert the 
same state functions into the alternative form of the path 
ensemble average, Eq. (pj|): (This is ultimately equiva- 
lent to switching and J-.) 



(22) 



This is the Kawasaki nonlinear response relation , 
applied to stochastic dynamics, and generalized to arbi- 
trary forcing. This relation can also be written in an 
explicitly renormalized form [|| by expanding the dissi- 
pative work as —AF + W, and rewriting the free energy 
change as a work average using the Jarzynski relation, 
Eq. (0). 

{fHr))),^{fme-nn/i^-nn (23) 

Simulation data indicates that averages calculated with 
the renormalized expression typically have lower statisti- 
cal errors [||. 

The probability distribution of a nonequilibrium en- 
semble can be derived from the Kawasaki relation, 
Eq. (p3|), by setting the state function to be = 
/(a:(r)) — S(^x — x{t)) , a S function of the state of the 
system at time r; 



Pncq{x,T\M) ^ p{x\P,E{t)) 



(24) 
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Here Pncq{x, t|M) is the nonequilibrium probability dis- 
tribution and p(a;|/3, E(r)) is the equiUbrium probabihty 
of the same state. The subscript 'x' indicates that the 
average is over all paths that start in state x. In contrast 
the lower average is over all paths starting from an equi- 
librium ensemble. Thus the nonequilibrium probability 
of a state is, to zeroth order, the equilibrium probability, 
and the correction factor can be related to a nonequilib- 
rium average of the work. 

There are several other far-from-equilibrium relations 
that have been derived from, or are related to the 
Kawasaki response. The transient time correlation func- 
tion (TTCF) |^,|6| gives another set of relations for 
the nonlinear response of a system, and are reputable 
of greater practical utility than the Kawasaki response 
relation. Unfortunately it appears that TTCF can not 
be applied to the systems considered in this paper, since 
a crucial step linking the two formalisms |^ makes the as- 
sumption that the dynamics are deterministic, and there- 
fore that only an average over initial conditions is needed. 
Similarly Evans and Morriss have derived several inter- 
esting relations for the heat capacity of a nonequilibrium 
steady-state , but again these relations are not gener- 
ally applicable because it is assumed that the probability 
of a trajectory is independent of the temperature of the 
heat bath. 



IV. CONCLUSIONS 

All of the relations derived in this paper are directly 
applicable to systems driven far-from-equilibrium. These 
relations follow if the dynamics are microscopically re- 
versible in the sense of Eq. (13). This relation was shown 
to hold if the dynamics are Markovian and balanced. Al- 
though I have concentrated on stochastic dynamics with 
discrete time and phase space, this should not be taken 
as a fundamental limitation. The extension to continu- 
ous phase space and time appears straightforward, and 
deterministic dynamics can be taken as a special case of 
stochastic dynamics. 
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